<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.11"/>
<title>GrPPI: Pipeline pattern</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
  $(document).ready(function() { init_search(); });
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="logo.svg"/></td>
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname">GrPPI
   &#160;<span id="projectnumber">0.2</span>
   </div>
   <div id="projectbrief">Generic and Reusable Parallel Pattern Interface</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.11 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li class="current"><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li><a href="modules.html"><span>Modules</span></a></li>
      <li><a href="namespaces.html"><span>Namespaces</span></a></li>
      <li><a href="annotated.html"><span>Classes</span></a></li>
      <li><a href="files.html"><span>Files</span></a></li>
      <li>
        <div id="MSearchBox" class="MSearchBoxInactive">
        <span class="left">
          <img id="MSearchSelect" src="search/mag_sel.png"
               onmouseover="return searchBox.OnSearchSelectShow()"
               onmouseout="return searchBox.OnSearchSelectHide()"
               alt=""/>
          <input type="text" id="MSearchField" value="Search" accesskey="S"
               onfocus="searchBox.OnSearchFieldFocus(true)" 
               onblur="searchBox.OnSearchFieldFocus(false)" 
               onkeyup="searchBox.OnSearchFieldChange(event)"/>
          </span><span class="right">
            <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
          </span>
        </div>
      </li>
    </ul>
  </div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Pipeline pattern </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>The <b>pipeline</b> pattern receives a data stream that is processed in several stages. Each stage processes the data produced by the previous stage and delivers its result to the next stage.</p>
<p>The interface to the <b>pipeline</b> pattern is provided by function <code><a class="el" href="group__pipeline__pattern__native.html#ga50396cad2879f1facd37757ea3aa564b" title="Invoke Pipeline pattern on a data stream with native parallel execution. ">grppi::pipeline()</a></code>. As all functions in <em>GrPPI</em>, this function takes as its first argument an execution policy.</p>
<div class="fragment"><div class="line"><a class="code" href="group__pipeline__pattern__native.html#ga50396cad2879f1facd37757ea3aa564b">grppi::pipeline</a>(exec, other_arguments...);</div></div><!-- fragment --><h2>Pipeline variants</h2>
<p>There are several variants:</p>
<ul>
<li><em>Standalone pipeline</em>: Is a top level pipeline. Invoking the algorithm runs the pipeline.</li>
<li><em>Composable pipeline</em>: Builds a pipeline object that can be later inserted into another pattern.</li>
</ul>
<h2>Key elements in pipeline</h2>
<p>The key elements in a <b>pipeline</b> are the <b>Generator</b> producing data items and the <b>Transformer</b> stages.</p>
<p>A <b>Generator</b> is any C++ callable entity that takes zero arguments and produces data items from a given type. Thus, a <b>Generator</b> <code>gen</code> is any operation that, given an output type <code>U</code>, makes valid the following:</p>
<div class="fragment"><div class="line">U res{gen()};</div><div class="line"><span class="comment">//...</span></div><div class="line"><span class="keywordflow">if</span> (res) { <span class="comment">/* ... */</span> }  <span class="comment">// If res holds a value</span></div><div class="line"><span class="keywordflow">if</span> (!res) { <span class="comment">/* ... */</span> } <span class="comment">// If res does not hold a value</span></div><div class="line"><span class="keyword">auto</span> value = * res;     <span class="comment">// Get value held by res</span></div></div><!-- fragment --><p>A <b>Transformer</b> is any C++ callable entity that takes a data item and transforms it. The input type and the output type may differ. Thus, a transformer <code>op</code> is any operation that, given an input <code>x</code> of type <code>T</code> and output type <code>U</code>, makes valid the following:</p>
<div class="fragment"><div class="line">U res {op(x)};</div></div><!-- fragment --><h2>Details on pipeline variants</h2>
<h3>Standalone pipeline</h3>
<p>A <em>standalone pipeline</em> generates data from a source and passes the output to the first stage that applies a transformation to each data item. The resulting items are passed to the next stage and so on.</p>
<p>Consequently, a pipeline with a <b>Generator</b> <code>gen</code> and <code>N</code> <b>Transformer</b> <em>stages</em> (<code>s1, s2, ..., sN</code>) performs the following computation:</p>
<div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno">    1</span>&#160;sN(...s2(s1(gen())))</div></div><!-- fragment --><p><b>Note</b>: Each stage may run concurrently with other stages. However, there are dependencies between stages, so that every item passes sequentially across stages. </p><hr/>
<p> <b>Example</b>: Generate a sequence of integers, apply consecutive transformations (from int to double and from double to string), and write to standard output. </p><div class="fragment"><div class="line"><span class="keywordtype">int</span> n = 100;</div><div class="line"><a class="code" href="group__pipeline__pattern__native.html#ga50396cad2879f1facd37757ea3aa564b">grppi::pipeline</a>(e, </div><div class="line">  [n]() -&gt; optional&lt;double&gt; { </div><div class="line">    <span class="keyword">static</span> <span class="keywordtype">int</span> x = 0;</div><div class="line">    <span class="keywordflow">if</span> (x&lt;n) <span class="keywordflow">return</span> x++;</div><div class="line">    <span class="keywordflow">else</span> <span class="keywordflow">return</span> {}; </div><div class="line">  },</div><div class="line">  [](<span class="keywordtype">double</span> x) { <span class="keywordflow">return</span> x*x; },</div><div class="line">  [](<span class="keywordtype">double</span> x) { <span class="keywordflow">return</span> 1/x; },</div><div class="line">  [](<span class="keywordtype">double</span> x) { cout &lt;&lt; x &lt;&lt; endl; }</div><div class="line">);</div></div><!-- fragment --><hr/>
<h3>Composable pipeline</h3>
<p>A <em>composable pipeline</em> returns a representation of the pipeline that can be used to perform declarative composition of streaming patterns. </p><hr/>
<p> <b>Example</b>: Use a farm to read intenger from a file and write to another file strings. The transformation phase is a pipeline that performs transformation in two stages. </p><div class="fragment"><div class="line"><a class="code" href="group__farm__pattern.html#ga26bdb7c1b6c3d4bedbcfc1774d00833c">grppi::farm</a>(ex1,</div><div class="line">  [&amp;input]() -&gt; optional&lt;int&gt; {</div><div class="line">    <span class="keywordtype">int</span> n;</div><div class="line">    input &gt;&gt; n;</div><div class="line">    <span class="keywordflow">if</span> (!input) <span class="keywordflow">return</span> {};</div><div class="line">    <span class="keywordflow">else</span> <span class="keywordflow">return</span> n;</div><div class="line">  },</div><div class="line">  <a class="code" href="group__pipeline__pattern__native.html#ga50396cad2879f1facd37757ea3aa564b">grppi::pipeline</a>(ex2,</div><div class="line">    [](<span class="keywordtype">int</span> x) -&gt; <span class="keywordtype">double</span> { <span class="keywordflow">return</span> func1(x); },</div><div class="line">    [](<span class="keywordtype">double</span> x) -&gt; <span class="keywordtype">string</span> { <span class="keywordflow">return</span> func2(x); },</div><div class="line">  ),</div><div class="line">  [&amp;output](<span class="keywordtype">string</span> s) {</div><div class="line">    output &lt;&lt; s &lt;&lt; endl;</div><div class="line">  }</div><div class="line">);  </div></div><!-- fragment --><hr/>
<p>This <em>composable pipeline</em> can also be used to build complex composed patterns in a non-declarative way. </p><hr/>
<p> <b>Example</b>: Piecewise construction of a farm of pipelines. </p><div class="fragment"><div class="line"><span class="keyword">auto</span> reader = [&amp;input]() -&gt; optional&lt;int&gt; {</div><div class="line">  <span class="keywordtype">int</span> n;</div><div class="line">  input &gt;&gt; n;</div><div class="line">  <span class="keywordflow">if</span> (!input) <span class="keywordflow">return</span> {};</div><div class="line">  <span class="keywordflow">else</span> <span class="keywordflow">return</span> n;</div><div class="line">};</div><div class="line"></div><div class="line"><span class="keyword">auto</span> processor = <a class="code" href="group__pipeline__pattern__native.html#ga50396cad2879f1facd37757ea3aa564b">grppi::pipeline</a>(ex2,</div><div class="line">  [](<span class="keywordtype">int</span> x) -&gt; <span class="keywordtype">double</span> { <span class="keywordflow">return</span> func1(x); },</div><div class="line">  [](<span class="keywordtype">double</span> x) -&gt; <span class="keywordtype">string</span> { <span class="keywordflow">return</span> func2(x); },</div><div class="line">);</div><div class="line"></div><div class="line"><span class="keyword">auto</span> writer = [&amp;output](<span class="keywordtype">string</span> s) {</div><div class="line">  output &lt;&lt; s &lt;&lt; <span class="stringliteral">&quot;\n&quot;</span>;</div><div class="line">};</div><div class="line"></div><div class="line"><a class="code" href="group__farm__pattern.html#ga26bdb7c1b6c3d4bedbcfc1774d00833c">grppi::farm</a>(ex1, reader, processor, writer);</div></div><!-- fragment --> </div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.11
</small></address>
</body>
</html>
